Dyna-style planning with linear function approximation and prioritized sweeping

We consider the problem of efficiently learning optimal control policies and value functions over large state spaces in an online setting in which estimates must be available after each interaction with the world. This paper develops an explicitly model-based approach extending the Dyna architecture to linear function approximation. Dyna-style planning proceeds by generating imaginary experience from the world model and then applying model-free reinforcement learning algorithms to the imagined state transitions. Our main results are to prove that linear Dyna-style planning converges to a unique solution independent of the generating distribution, under natural conditions. In the policy evaluation setting, we prove that the limit point is the least-squares (LSTD) solution. An implication of our results is that prioritized-sweeping can be soundly extended to the linear approximation case, backing up to preceding features rather than to preceding states. We introduce two versions of prioritized sweeping with linear Dyna and briefly illustrate their performance empirically on the Mountain Car and Boyan Chain problems

Future research directions in the machining of Inconel 718

Inconel 718 is the most popular nickel-based superalloy, extensively used in aerospace, automotive and energy industries owing to its extraordinary thermomechanical properties. It is also notoriously a difficult-to-cut material, due to its short tool life and low productivity in machining operations. Despite significant progress in cutting tool technologies, the machining of Inconel 718 is still considered a grand challenge.This paper provides a comprehensive review of recent advances in machining Inconel 718. The progress in cutting tools’ materials, coatings, geometries and surface texturing for machining Inconel 718 is reviewed. The investigation is focused on the most adopted tool materials for machining of Inconel 718, namely Cubic Boron Nitrides (CBNs), ceramics and coated carbides. The thermal conductivity of cutting tool materials has been identified as a major parameter of interest. Process control, based on sensor data for monitoring the machining of Inconel 718 alloy and detecting surface anomalies and tool wear are reviewed and discussed. This has been identified as the major step towards realising real-time control for machining safety critical Inconel 718 components. Recent advances in various processes, e.g. turning, milling and drilling for machining Inconel 718 are investigated and discussed. Recent studies related to machining additively manufactured Inconel 718 are also discussed and compared with the wrought alloy. Finally, the state of current research is established, and future research directions proposed.<br/

A human lung tumor microenvironment interactome identifies clinically relevant cell-type cross-talk.

BackgroundTumors comprise a complex microenvironment of interacting malignant and stromal cell types. Much of our understanding of the tumor microenvironment comes from in vitro studies isolating the interactions between malignant cells and a single stromal cell type, often along a single pathway.ResultTo develop a deeper understanding of the interactions between cells within human lung tumors, we perform RNA-seq profiling of flow-sorted malignant cells, endothelial cells, immune cells, fibroblasts, and bulk cells from freshly resected human primary non-small-cell lung tumors. We map the cell-specific differential expression of prognostically associated secreted factors and cell surface genes, and computationally reconstruct cross-talk between these cell types to generate a novel resource called the Lung Tumor Microenvironment Interactome (LTMI). Using this resource, we identify and validate a prognostically unfavorable influence of Gremlin-1 production by fibroblasts on proliferation of malignant lung adenocarcinoma cells. We also find a prognostically favorable association between infiltration of mast cells and less aggressive tumor cell behavior.ConclusionThese results illustrate the utility of the LTMI as a resource for generating hypotheses concerning tumor-microenvironment interactions that may have prognostic and therapeutic relevance

Intelligent Cooperative Control Architecture: A Framework for Performance Improvement Using Safe Learning

Planning for multi-agent systems such as task assignment for teams of limited-fuel unmanned aerial vehicles (UAVs) is challenging due to uncertainties in the assumed models and the very large size of the planning space. Researchers have developed fast cooperative planners based on simple models (e.g., linear and deterministic dynamics), yet inaccuracies in assumed models will impact the resulting performance. Learning techniques are capable of adapting the model and providing better policies asymptotically compared to cooperative planners, yet they often violate the safety conditions of the system due to their exploratory nature. Moreover they frequently require an impractically large number of interactions to perform well. This paper introduces the intelligent Cooperative Control Architecture (iCCA) as a framework for combining cooperative planners and reinforcement learning techniques. iCCA improves the policy of the cooperative planner, while reduces the risk and sample complexity of the learner. Empirical results in gridworld and task assignment for fuel-limited UAV domains with problem sizes up to 9 billion state-action pairs verify the advantage of iCCA over pure learning and planning strategies

IT controls in the public cloud : success factors for allocation of roles and responsibilities

The rapid adoption of cloud computing by organizations has resulted in the transformation of the roles and responsibilities of staff in managing the information technology (IT) resources (via IT governance controls) that have migrated to the cloud. Hence, the objective of this research is to provide a set of success factors that can assist IT managers to allocate the roles and responsibilities of IT controls appropriately to staff to manage the migrated IT resources. Accordingly, we generated a set of success factors from behavioral and information systems (IS) literature. These success factors were verified using in-depth interviews of executives from the United Arab Emirates (UAE). The empirical intervention suggests that the role allocation is driven predominantly by people’s skills, competencies, organizational strategy, structures, and policies. In addition, the research made clear that the most significant competency and skill for a person allocated to IT controls is to be able to evaluate and manage a cloud service provider, especially in terms of risks, compliance, and security issues related to public cloud technology. The findings of this study not only offer new insights for scholars and practitioners involved in assigning responsibilities but also provide extensions for IT governance framework authorities to align their guidelines to the emerging cloud technology

Understanding teaching assistant self-efficacy in role and in training: its susceptibility to influence

There has been a noted growth in the number of teaching assistants (TAs) in mainstream schools (DfE, 2013a). Research is inconclusive about their efficacy at changing outcomes for children (Alborz et al 2009; Blatchford et al, 2009) and has proposed more training for TAs (Russell et al, 2005). Generic training models have suggested that enhancing self-efficacy in turn improves performance. This exploratory study investigated factors that may influence TAs’ sense of self-efficacy and its susceptibility to influence in training. Following two modes of mode of school-based training by Educational Psychologists (EPs) data were collected from 14 mainstream secondary school TAs using focus groups. A thematic analysis noted themes regarding self-efficacy, aligned with Bandura’s (1977) sources of information, outcome expectations and whole school support and norms. Review of the data is likely to be able to guide potential trainers to coach consult strategies which are self-efficacy supportive and which address contextual factors including the perceived status of TAs in schools

Manipulation of Signaling Thresholds in “Engineered Stem Cell Niches” Identifies Design Criteria for Pluripotent Stem Cell Screens

In vivo, stem cell fate is regulated by local microenvironmental parameters. Governing parameters in this stem cell niche include soluble factors, extra-cellular matrix, and cell-cell interactions. The complexity of this in vivo niche limits analyses into how individual niche parameters regulate stem cell fate. Herein we use mouse embryonic stem cells (mESC) and micro-contact printing (µCP) to investigate how niche size controls endogenous signaling thresholds. µCP is used to restrict colony diameter, separation, and degree of clustering. We show, for the first time, spatial control over the activation of the Janus kinase/signal transducer and activator of transcription pathway (Jak-Stat). The functional consequences of this niche-size-dependent signaling control are confirmed by demonstrating that direct and indirect transcriptional targets of Stat3, including members of the Jak-Stat pathway and pluripotency-associated genes, are regulated by colony size. Modeling results and empirical observations demonstrate that colonies less than 100 µm in diameter are too small to maximize endogenous Stat3 activation and that colonies separated by more than 400 µm can be considered independent from each other. These results define parameter boundaries for the use of ESCs in screening studies, demonstrate the importance of context in stem cell responsiveness to exogenous cues, and suggest that niche size is an important parameter in stem cell fate control